The Mario AI Championship 2009-2012

benchmarks. In reinforcement learning, the choice has long stood between simplistic toy problems such as pole balancing and the Mountain Car, and complex, slow and nonreplicable robot problems. Within the CI/AI in games community, a series of competitions has grown up where competitors submit controllers for modified or reconstructed versions of existing computer games. Using existing computer games as AI benchmarks brings several benefits, the most important being that the games are almost guaranteed to contain interesting AI challenges by virtue of being popular among human players. (One of the most important reasons games are engaging to humans is that they provide learning challenges [Koster 2005]). Almost as important is that good scoring mechanisms are available, that the visual aspects of the games make it easy to compare and characterize the performance of the controllers, and that it is easy to engage both students and the general public in the competition. Several recently introduced competitions are based on games such as Ms. Pac-Man (Lucas 2007), the first-person shooter Unreal Tournament (Hingston 2010), the real-time strategy game StarCraft, and the car racing game TORCS (Loiacono et al. 2010). In 2009, Julian Togelius and Sergey Karakovskiy set out to create a benchmark for game AI controllers based on Infinite Mario Bros (IMB). IMB is an open source clone (created by Markus Persson, who later went on to create Minecraft) of Nintendo’s platform game Super Mario Bros. (SMB), which has been one of the world’s most influential games since its release in 1985. The core gameplay task in IMB, like in SMB, is to guide the player character Mario from the start to the end of a two-dimensional world without getting killed by enemies or falling down gaps, and while collecting coins and powerups. Unlike SMB, IMB features in-game procedural generation of levels, thus the word Infinite in its title. Creating the first version of the Mario AI Benchmark software involved significant reengineering of the core loops of the game, making all timing optional (so that the benchmark can run several thousands times faster than the original game Competition Report